[feature] Add LSP XQuery function module#6130
[feature] Add LSP XQuery function module#6130joewiz wants to merge 11 commits intoeXist-db:developfrom
Conversation
Switch from atom-editor-support (GET-based, single error, client-side
import resolution) to language-support (POST JSON, multi-error diagnostics,
server-side import resolution via lsp:* functions).
Key changes:
- linting.ts: POST /api/diagnostics with structured JSON response
- analyzed-document.ts: POST /api/{completions,hover,definition} with
full query text; removed resolveImports/parseImports/getParameters
- server.ts: pass document text to getCompletions and getHover
- utils.ts: detect language-support XAR instead of atom-editor
Requires eXist-db 7.0+ with lsp:* module (eXist-db/exist#6130).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
To be honest, I am excited about this library! Or maybe we can have it as an integral part of exist-db from now on but provide an additional XAR to retrofit this on older versions (I am thinking of the current version 6). |
| } | ||
|
|
||
| private Sequence buildFunctionHover(final FunctionSignature sig, | ||
| final UserDefinedFunction udf) throws XPathException { |
|
I really like the idea. One question concerns documentation. I would like to know which parts of the lsp protocol are implemented and what the selection process was? If we bundle this with core, we would need to document this in the main docs. Which brings me to my other question. I wonder if tight integration with core is really the right way to go. We already have two potential consumers eXide and ide s connecting via atom plugin. Both expath packages would be tightly coupled to exist version with respect to available LSP implementation. It seems more intuitive to have them depend on an expath library, we include by default. But I m not very firm on this. What do you think? |
…ompletions, hover, and definition Add a new internal XQuery module (http://exist-db.org/xquery/lsp) that exposes eXist-db's XQuery compiler internals for Language Server Protocol support. Functions: - lsp:diagnostics($expr, $module-load-path?) — compiles XQuery and returns an array of diagnostic maps (line, column, severity, code, message) - lsp:symbols($expr, $module-load-path?) — compiles XQuery and returns an array of document symbol maps (name, kind, line, column, detail) - lsp:completions($expr, $module-load-path?) — returns an array of completion item maps (label, kind, detail, documentation, insertText) including built-in functions, keywords, and user-declared symbols - lsp:hover($expr, $line, $column, $module-load-path?) — returns hover info (contents, kind) for the symbol at the given position - lsp:definition($expr, $line, $column, $module-load-path?) — returns the definition location (line, column, name, kind) for user-declared functions and variables 68 tests covering all five functions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
joewiz
force-pushed
the
feature/lsp-module
branch
from
d78d1ae to
796d1ae
Compare
When the symbol at the cursor resolves to a function defined in an imported library module, the result map now includes a "uri" key containing the source path of that module. This enables IDE clients to open the target module file and jump to the definition. Uses func.getSource().path() to detect when the definition lives in a different module than the input expression. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the XQuery parser found syntax errors, Diagnostics.java passed -1, -1 as line/column (which became 0, 0 after Math.max). The parser stores the actual exception with correct line/column — extract it from parser.getLastException() as RecognitionException or XPathException. Adds test for multi-line error positioning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New function that finds all references to the symbol at a given position. Walks the compiled AST collecting all FunctionCall or VariableReference nodes that resolve to the same definition. Returns an array of maps with line, column, name, and kind. Includes the declaration itself. Enables "Find All References" and "Rename Symbol" in IDE clients. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…uence The function signature declares array(*) return type but was returning a ValueSequence of maps, causing "Expected cardinality: exactly one, got N" errors. Wrap results in ArrayType to match the declaration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wrap the entire hover lookup in a try/catch so that positions on whitespace, operators, or other non-symbol locations return an empty sequence instead of propagating exceptions to the HTTP layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ompletions, hover, and definition Add a new internal XQuery module (http://exist-db.org/xquery/lsp) that exposes eXist-db's XQuery compiler internals for Language Server Protocol support. Functions: - lsp:diagnostics($expr, $module-load-path?) — compiles XQuery and returns an array of diagnostic maps (line, column, severity, code, message) - lsp:symbols($expr, $module-load-path?) — compiles XQuery and returns an array of document symbol maps (name, kind, line, column, detail) - lsp:completions($expr, $module-load-path?) — returns an array of completion item maps (label, kind, detail, documentation, insertText) including built-in functions, keywords, and user-declared symbols - lsp:hover($expr, $line, $column, $module-load-path?) — returns hover info (contents, kind) for the symbol at the given position - lsp:definition($expr, $line, $column, $module-load-path?) — returns the definition location (line, column, name, kind) for user-declared functions and variables 68 tests covering all five functions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- lsp:definition() now resolves function calls across imported modules by walking the import tree and matching namespace URI + local name - Add lsp:references() for find-all-references support, returning an array of location maps with uri, line, and column - Fix lsp:references() return type to use ArrayType
- Fix lsp:diagnostics() returning line 0 for parser errors by extracting line numbers from XPathException messages - Prevent lsp:hover() from throwing 500 for non-symbol positions by returning empty sequence instead of NPE
…ntax - Diagnostics: convert parser's 1-indexed lines/columns to 0-indexed for LSP protocol compliance - ReferencesTest: use array:size() and $refs(n) syntax instead of count() and $refs[n] — lsp:references returns an XQuery array Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…/lsp-module # Conflicts: # exist-core/src/main/java/org/exist/xquery/functions/lsp/Diagnostics.java # exist-core/src/test/java/org/exist/xquery/functions/lsp/ReferencesTest.java
|
@line-o @duncdrum I am happy to adapt this to an EXPath Package, with the goal of freeing this feature from being tied to eXist releases. Is there a package repo you could suggest that I should model this new package on, in terms of build and test harness tooling and infrastructure? Or if not, could you describe the approach you'd suggest? |
|
[This response was co-authored with Claude Code. -Joe] @line-o @duncdrum Thank you both for the feedback — I agree that an EXPath Package is the right approach. Here's the plan we've developed: Migration Plan:
|
| Function | LSP Method | Why |
|---|---|---|
lsp:diagnostics |
textDocument/publishDiagnostics |
Multi-error compile checking — replaces util:compile-query which returns only 1 error |
lsp:symbols |
textDocument/documentSymbol |
Functions + variables with types — enables outline/breadcrumbs |
lsp:completions |
textDocument/completion |
All built-in + user-declared functions — includes imported modules |
lsp:hover |
textDocument/hover |
Function signatures + docs at cursor |
lsp:definition |
textDocument/definition |
Jump to function/variable declaration, including cross-module (returns module URI) |
lsp:references |
textDocument/references |
Find all usages of a function/variable — enables rename |
These were chosen as the foundational LSP capabilities that give the most value with the least API surface. All 6 functions use the same pattern: compile the query text, walk the expression tree, return structured results. Future additions (signatureHelp, codeAction, workspace/symbol) can be added incrementally.
Current consumers
- eXide (PR eXist-db/eXide#778) — uses dynamic
function-lookupforlsp:*, so it works whether the functions come from core or from a package - existdb-langserver (PR wolfgangmm/existdb-langserver#68) — calls REST endpoints, unaffected by where the Java lives
Open questions
-
Repo location: Should this live under
eXist-db/existdb-language-support(the org), or start on my fork and PR later? Is there a template repo for EXPath packages with Java components? -
Distribution bundling: Should the XAR be included in the default eXist-db distribution (like monex), or installed separately? Including it by default would give all users LSP support out of the box.
-
Java version targeting: The current code uses Java 21 features (pattern matching for
instanceof, text blocks). Should we target Java 17 for broader compatibility with eXist-db 6.x, or is Java 21 acceptable? -
CI/testing: What CI setup do you recommend for EXPath packages? The Java unit tests need an embedded eXist-db instance (
ExistXmldbEmbeddedServer). Should tests run against multiple eXist-db versions? -
Namespace: Is
http://exist-db.org/xquery/lspthe right namespace for an EXPath package, or should it use a different convention (e.g.,http://exist-db.org/xquery/lspis fine since monex useshttp://exist-db.org/xquery/console)?
Happy to proceed with the migration once we've aligned on these questions.
|
[This response was co-authored with Claude Code. -Joe] Closing this PR in favor of packaging the LSP module as a standalone EXPath package (XAR) per reviewer feedback. This allows:
The standalone package is available at: https://github.com/joewiz/exist-lsp Same 6 functions ( |
4 participants
Summary
Adds a new internal XQuery module (
http://exist-db.org/xquery/lsp) that exposes eXist-db's XQuery compiler internals for Language Server Protocol support. This enables editor tooling (such as eXide) to provide hover documentation, go-to-definition, completions, diagnostics, and document symbols using the server's own compiler knowledge.Functions
lsp:diagnostics($expr, $module-load-path?)— compiles XQuery and returns an array of diagnostic maps (line,column,severity,code,message)lsp:symbols($expr, $module-load-path?)— returns an array of document symbol maps (name,kind,line,column,detail) for all declared functions and variableslsp:completions($expr, $module-load-path?)— returns completion items (label,kind,detail,documentation,insertText) including built-in functions, keywords, and user-declared symbolslsp:hover($expr, $line, $column, $module-load-path?)— returns hover info (contents,kind) for the symbol at the given positionlsp:definition($expr, $line, $column, $module-load-path?)— returns the definition location (line,column,name,kind) for user-declared functions and variablesWhat Changed
exist-core/src/main/java/org/exist/xquery/functions/lsp/LspModule.java— module descriptor registering all five functionsexist-core/src/main/java/org/exist/xquery/functions/lsp/Diagnostics.java— compile-check implementationexist-core/src/main/java/org/exist/xquery/functions/lsp/Symbols.java— document symbols via AST walkexist-core/src/main/java/org/exist/xquery/functions/lsp/Completions.java— completions from function library + user ASTexist-core/src/main/java/org/exist/xquery/functions/lsp/Hover.java— hover info via position lookupexist-core/src/main/java/org/exist/xquery/functions/lsp/Definition.java— go-to-definition via AST position lookupexist-distribution/src/main/config/conf.xml+ all testconf.xmlfiles — module registered underorg.exist.xquery.functions.lsp.LspModuleIntegration Testing
All five functions have been integrated into eXide and tested end-to-end against a running eXist-db instance with this module deployed. See joewiz/eXide#17 for the eXide side of this work.
lsp:*functionslsp:diagnostics()drives the compile-check (squiggles) in the editorTest Plan
mvn test -pl exist-core -Dtest="lsp.DiagnosticsTest,lsp.SymbolsTest,lsp.CompletionsTest,lsp.HoverTest,lsp.DefinitionTest"— 68/68 passlsp:diagnostics()returns errors with correct line/column for invalid XQuery — verified via compile check in eXidelsp:symbols()returns function and variable declarations with correct positions — verified via outline panel and Navigate → Symbol in eXidelsp:completions()includes built-in functions and user-defined local functions — verified via Ctrl+Space completions in eXidelsp:hover()returns signature and documentation for functions at a given position — verified via hover tooltip in eXidelsp:definition()returns source location for user-defined functions and variables — verified via Go to Definition in eXide🤖 Generated with Claude Code